Add 0084 orphan fallback (both observers)#241
Merged
Conversation
Proposal 0084 (nested-fan-out span lineage), second of the Wave 3 PRs; completes the observer-side work started in #240. Generalize the observability orphan fallback -- a provider or framework call with no open calling-node span (a guardrail or middleware side call) -- in both the OTel and Langfuse observers to resolve the nearest enclosing wrapper on the calling node's lineage, chain-routed to the correct inner fan-out instance. A shared resolver backs both the node parent and the orphan fallback in each observer, and now also backs FailureIsolatedEvent parenting so the marker resolves consistently across observers. The Langfuse node-observation key gains the enclosing lineage chains (mirroring the OTel node-span key from #240) so nested exact-match works there too. Un-defer observability fixtures 133 (OTel orphan) and 134 (Langfuse, exact-match + orphan). The orphan-call primitive calls_llm_from_wrapper is modeled on NodeSpec so 133 and 134 both parse and run. Known limitation: a wrapper call issued before the wrapped node body (pre-phase) inside a nested fan-out resolves to the outer instance in Langfuse, which creates instance observations lazily, while OTel resolves to the inner instance -- a cross-observer parity gap the fixtures do not cover. The conformance.toml 0084 note documents it; the Langfuse eager-creation fix and the spec question are tracked as follow-ups.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR completes the observer-side implementation of proposal 0084 (nested fan-out span/observation lineage) by adding a chain-aware “orphan fallback” parent resolver in both OTel and Langfuse, updating Langfuse node observation keying to include lineage chains, and enabling the previously deferred conformance fixtures (133/134) via a new conformance adapter directive for wrapper-issued LLM calls.
Changes:
- Implement shared chain-aware “nearest enclosing wrapper” resolution for orphaned provider calls (and align FailureIsolatedEvent parenting via the same resolver).
- Extend Langfuse in-flight node-observation keying with lineage chains to avoid collisions under nested concurrency.
- Un-defer and wire new conformance harness runners and directive parsing for wrapper-issued (orphan) LLM calls.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_observability_otel.py | Updates unit coverage for OTel orphan fallback to include lineage chains. |
| tests/unit/test_observability_langfuse.py | Updates Langfuse unit coverage for lineage-aware node keys and orphan fallback. |
| tests/conformance/test_observability.py | Enables fixture 133 and adds a dedicated runner to exercise nested orphan fallback in OTel. |
| tests/conformance/test_observability_langfuse.py | Enables fixture 134 with a dedicated runner comparing Langfuse vs OTel resolved parents. |
| tests/conformance/test_fixture_parsing.py | Removes parsing deferrals for 133/134 now that the directive is modeled and runnable. |
| tests/conformance/harness/directives.py | Adds CallsLlmFromWrapperSpec and NodeSpec.calls_llm_from_wrapper (additive directive). |
| src/openarmature/observability/otel/observer.py | Replaces ad-hoc fallbacks with a shared chain-aware enclosing-wrapper parent resolver. |
| src/openarmature/observability/langfuse/observer.py | Adds lineage chains to node keys; introduces shared enclosing-wrapper resolver; updates FailureIsolatedEvent parenting. |
| src/openarmature/graph/events.py | Updates FailureIsolatedEvent documentation to reflect §5.5 orphan fallback parenting. |
| conformance.toml | Marks proposal 0084 as implemented and documents the known Langfuse pre-phase parity limitation. |
Add unit-test guards pinning the branch-side of the §5.5 orphan fallback in both observers -- an orphan provider call inside a parallel branch parents under the per-branch dispatch span / observation (the nearest enclosing wrapper), the branch analog of the fan-out orphan tests. The 0084 fixtures cover fan-out nesting only; a spec fixture for the branch case is flagged for the batched review (release-v0.17.0 coord).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second and final observer-side PR for proposal 0084 (nested-fan-out span lineage), completing the work started in #240. Under the current spec pin, no bump.
What
_resolve_enclosing_wrapper_contexton OTel,_resolve_enclosing_wrapper_observation_idon Langfuse) backs both the node parent and the orphan fallback, so an in-body call and a wrapper call on the same lineage resolve to the same parent.FailureIsolatedEventparenting now routes through it too, so the marker resolves consistently across the two observers.calls_llm_from_wrapperorphan-call primitive is modeled onNodeSpec, so 133/134 both parse/round-trip and run.Known limitation (documented, tracked)
A wrapper call issued in the pre-phase (before the wrapped node body) inside a nested fan-out resolves to the outer instance in Langfuse — which creates instance observations lazily — while OTel resolves to the inner instance. That's a cross-observer parity gap for the pre-phase scenario, which the fixtures do not cover (the harness exercises the orphan path via post-phase instance middleware, where the two agree; 134 asserts that agreement with a live OTel-vs-Langfuse parent comparison). The
conformance.toml0084 note documents it, and the Langfuse eager-instance-observation fix plus the spec question are tracked as follow-ups.Review
This PR went through a multi-lens adversarial review, which surfaced the pre-phase parity gap above (previously masked), a cross-observer asymmetry in failure-isolation parenting (fixed here), and a tautological parity assertion in fixture 134 (replaced with a real comparison).
Testing
pytest tests/— 1776 passed. ruff + pyright clean.